ORIGINAL

Niger J Paed 2015; 42 (2):132 –136

Garba MA

Mustapha GM

Rejoice C

Mustapha B

Alhaji MA

Bello IA

Competency-based Learning: The

effectiveness of targeted resident

education and clinical auditing

feedback on completed death

certificate accuracy rates

Hassan AE

DOI:http://dx.doi.org/10.4314/njp.v42i2.12

Accepted: 9th June 2014

Abstract: Background: Death

certification is an integral part of

physician practice, yet common

errors are being encountered from

this vital source of health infor-

mation. Most medical training

programs lack formal curricula for

this important skill. Accurate in-

formation from DC will help in

improve our mortality data which

can be use public health purposes

Objectives: This study evaluated

the effect of a multifaceted educa-

tional intervention on accuracy of

completing death certification in a

tertiary-based paediatric residency

program

back sessions). Primary outcome

was the difference in scores pre-

and post-intervention and also the

rate of accuracy on the MAHI

death certificate scoring system.

Results: The mean score before the

intervention was 6.8±2.7 and

7.1±2.3 in both the intervention

and control cohorts respectively.

The mean score after the interven-

tion was 16.3±2.5 and 7.3±2.8 in

both the intervention and control

cohorts respectively indicating an

increase in scores. The mean dif-

ference in pre- and post-tutorial

scores was significant (t=20.39,

p=0.0001).

(

)

Garba MA

Mustapha GM, Rejoice C, Mustapha B,

Alhaji MA, Bello IA, Hassan AE

Department of Paediatrics,

University of Maiduguri Teaching

Hospital

PMB 1069 Maiduguri,

Borno state, Nigeria,

Email: ashirugarba10@gmail.com

Method: A pre-post intervention

and control cohort study over 12-

month period to assess the effect

of our multifaceted intervention

accuracy rate of completed DC

was conducted. The intervention

consisted of a 3-part program

Conclusions: We found that using

a multifaceted educational inter-

vention to train junior physicians

on how to correctly complete a DC

was effective in a residency-based

pediatric program

(

rial, presentation by a skilled

instructor, skills workshop, and

clinical mortality/auditing feed-

distribution of educational mate-

Keywords: Death certificate,

medical education, multifaceted

training, heath information,

Nigeria.

Introduction

in many medical school and residency programs are

give₁_,n₄_-₁l₀ittle if any formal training on how to complete

The death certificate (DC) is an important legal and pub-

lic health document issued by a hospital that declares the

date, location, and cause of a person’s death. Data from

death certificates constitute an essential component of

national vital statistics. Death certificates are used by

public health researchers for identification of the leading

causes of death, disease outbreaks and for surveillance

DC

. We found no published curricula at the under-

graduate medical and residency trainings that teach ac-

curate DC completion, although the Federal Ministry of

Health lists DC as a source of vital statistics for the na-

tion. Furthermore, the Medical and Dental council of

Nigeria (MDCN) evaluation form for the interns failed

to capture death certification skill as one o₁f_,₃_-t₉he core

on ac-

1

,2

of disease patterns . They are also used to determine

competencies to be assessed. Several studies

1

-

public health funding and clinical research priorities.

curate DC reports educational interventions that were

found to be benefit, however, the results were quite vari-

able and success was often non significant and short

3

More so in our setting where practice of performing

autopsies is rare, DC have become an even more impor-

tant source of data on the mortality. Studies have shown

that DC error rates are high, particularly in academic

setting w_-₆here junior physicians are the sole certifying

6

-9

lasting . To achieve a lasting change in residents' accu-

racy in completing DC, the resident must go through the

generic stages of behaviour change as described by Pro-

chaska: pre-contemplative; contemplative; preparation;

3

officers . Despite this responsibility, junior physicians

1

33

1

action; and maintenance or relapse . It is unusual for a

single intervention to be sufficient to move an individual

through all these stages and thus facilitate a lasting be-

havior change. Studies that comb₁i₁n_-₁e₃interventions have,

Educational intervention

Predisposing intervention

The first part of the interventions involved distribution

of educational material containing guidelines and rec-

ommendations on how to complete DC based on inter-

however, been more successful.

Such studies may

combine a predisposing intervention (designed to

achieve the preparation stage of change) with an ena-

bling intervention (designed to achieve the action stage

of change) and are in forcing intervention (designed to

achieve the maintenance stage and prevent the relapse

1

6

national classification of disesases-10 (ICD-10) .

Enabling intervention

14

stage of change) . The aim of this study was to deter-

mine whether a targeted junior physician (intern and

residents) education program would improve the quality

of DC completed by Pediatrics physicians. Research in

medical education suggests that optimal learning in-

volves the use of different methods of teaching and that

a multifaceted approa₁c₁h_-₁₃leads to higher retention of

The interns/residents attended a two-hour workshop on

how to complete DC. The workshop, moderated by two

of the authors (GMA and MMG), was held during the

usual time of the weekly academic program. The work-

shop was designed to focus on usefulness of DC, defini-

tion of terms and guidelines for completion of DC based

on ICD-10 . The workshop began with a brainstorming

session on 'what went wrong' and 'what went right' dur-

ing DC completion in which participants were asked to

identify reasons why DC that they have completed or

reviewed as satisfactory or otherwise. Each item on DC

was introduced by one of the instructors. Following this

participants were given clinical case-scenario of DC. An

example of a scenario is given below:

1

6

knowledge and skills.

We therefore, hypothesized

that a 3-part program (distribution of educational mate-

rial, presentation by a skilled instructor(s), skills/training

workshop, and clinical mortality/auditing feedback ses-

sions) would be effective in increasing the quality of DC

completed by interns/resident physicians. Accurate in-

formation from DC will help in improve our mortality

data which can be use public health and legal purposes

"9 month-old female with Down’s syndrome managed

for severe bronchopneumonia with background ven-

tricular septal defect died on day 3 of admission. What

was the underlying cause of death?”

Methods

Design

Based on the scenario participants were asked to per-

form the relevant tasks for their case, e.g., completing

underline cause of death for their scenario. When all

tasks were completed each completed DC was analyzed

and graded for accuracy, feedback was given. The ses-

sion was highly interactive, the faculty described ways

of using reflection and feedback ('what went right' and

'what went wrong') with reasons so as to improve accu-

racy of DC. The session ended with a review of the lit-

erature on morning presentation and participants were

provided with a memorandum on how to complete DC

(predisposing intervention).

We conducted an Institutional Review Board approved,

cohort pre-post comparison study over a 12-month pe-

riod to assess the effect of a targeted interns/residents

physician educational program on the quality of DC

completed by junior physicians (intern and residents). A

6

-month pre-intervention assessment period (January to

June 2013) was followed by the intervention educational

presentation (July 2013), weekly clinical mortality/

auditing and feedback sessions (July to December 2013)

and a 3-month post-intervention evaluation period

(

October to December 2013). The intervention was tar-

Reinforcing intervention

geted primarily at interns and residents at one tertiary-

based Pediatrics residency program. In an effort to avoid

psychological phenomenon (observational bias) that

produces an improvement in human behavior or per-

formance as a result of increased attention₅from superi-

Over the 3 months following the workshop, the course

all the authors provided feedback to internists/ residents

on the DC certified during routine weekly morbidity and

mortality sessions. These short discussions addressed

questions or concerns rose by the residents or brought to

light by the certifying physician, and were designed to

reinforce the elements taught earlier in the workshop.

1

ors or colleaguesbias (Hawthorne effect) we did not

detail our plan to audit the DCs before and after the in-

tervention to the participating interns/residents. Fifty

completed DC from the pediatrics department were ran-

domly selected for the last 6 month before the interven-

tion as pre-intervention cohort and 50 completed DC 3

month after/into the starts of the intervention as post-

intervention cohort. All DC completed by interns and

residents during the pre- and post comparison period

were included in the study. We excluded 7 DC which

did not have the designation of the certifying physicians.

Also DC completed during the same period from inter-

nal medicine department were evaluated as control co-

hort (residents from internal medicine received no addi-

tional training on DC).

Data collection

We used a standard DC approved by the WHO, which is

1

6

in use across the world . Part (I) outlines the events

from immediate events to the underlying cause of death

in a descending order. Part (II) outlines the associated co

-morbidities that add to the disease process. The DC

were graded and analyzed based on a modified version

of the Mid American Heart Institute (MAHI) Death Cer-

1

,9

tificate Scoring System used in previous studies . Each

item on scoring instrument was scored on a value of 0 to

1

34

2

, with 22 being the highest possible score. A score of

tory score in the post-intervention control cohort. The

overall mean score of improvement attributable to the

multifaceted educational improvement is shown in fig-

ure 1.

zero meant inaccurate information, 1 was partially cor-

rect and 2 fully correct. Each Completed DC’s score was

summed and labeled as satisfactory (≥19) or unsatisfac-

tory (≤18). Two of the authors (GMA and AIB) blinded

to the DC’s cohort reviewed all death certificates, and

the mean was considered as the final score for each par-

ticipant. The interrater reliability (rho) score of 0.93 was

obtained suggesting high level of agreement among the

assessors. Where there was marked discrepancy of more

than 4 points among the assessors, agreement was

achieved through reevaluation by both assessors to-

gether. After information was extracted on the DC,

scores were entered into the computerized spread sheets

Table 1: Modified Mid America Heart Institute Death Certifi-

cate Scoring System

Item Description

1

2

Etiologically relevant cause of death has been identified

Sequential format for Part I of death certificate has been

followed

3

4

5

6

7

Line A in Part I has been correctly identified

Line B in Part I has been correctly identified

Line C in Part I has been correctly identified

Only one condition per line has been identified

Co-morbid conditions have been correctly identified in

Part II of death certificate

(

Microsoft Excel version 2003, Redmond, Washington).

These results represented the final analysis/interpretation

of the DCs. For purposes of this study, we defined an

accurate DC as one that was both “satisfactory” as de-

fined above and included name of the certifying physi-

cians.

8

9

1

Did not utilize mechanistic terminal events

Did not utilize symptoms and signs

Did not oversimplify inappropriately

Did not report abbreviations

0

1

Data Analysis

Fig 1: Mean difference in the completed death certificate scores before

and after in the control and intervention cohorts (p<0.000)

Death certificates were classified categorically by the

absolute score obtained as satisfactory and unsatisfac-

tory. Fisher’s exact test was used for comparisons in-

volving the categories/items of the DC score before and

after the intervention. Mean DC score for pre and post

intervention for the two groups were compared using

paired t-test and the effect size was calcul₇ated based on

1

the method described by Cohen in 1988. All analyses

were one-sided and a p-value of < 0.05 was considered

to be statistically significant. Pre and post-intervention

assessment results were also tabulated. Statistical analy-

ses were performed using Microsoft Excel and SPSS

Table 2: The effect of the educational intervention on mean

death certificate scores before and after in the control and

intervention cohorts

1

6.0 software (Chicago)

Mean Scores

Interv_*ention

Effect Size P value

(d)

*

After

Before

Intervention

Group

Results

6.8±2.7 16.3±2.5

3.51

0.08

<0.001

1.000

A total of 200 DCs were analyzed (50 each before and

after the intervention in both the intervention and control

groups). In the DCs analyzed, the mean difference in the

pre and post-intervention scores in the intervention co-

hort was statistically significant (t=23.39, p < 0.001) and

also the mean difference between post-intervention

scores for the intervention and control cohorts was sig-

nificant (t=22.14, p< 0.001). While the mean difference

in the pre and post-intervention in the control cohort was

not statistically significant. This indicates an increase in

the scores of about 34% after the intervention both verti-

cally and horizontally (table 2) and (fig 1). Table 3 dem-

onstrates that the intervention cohort showed a signifi-

cant improvement in 7 of the 8 parameters of the DC

scored, compared to the control group were no such im-

provement was observed based on modified MAHI DC

Control Group 7.1±2.3 7.3±2.8

$

*

=mean ±0.SD, d= effect size, P value < 0.05= statistically

significant,

Table 3: Descriptions and frequency of Accuracy Rate in the

death certificate before and after the education intervention in

the control and intervention cohorts

Performance Parameters

Intervention

Before After

%) (%)

Etiologically relevant cause of death has been identified

P value

(

Control

Intervention

Sequence format for Part I of DC has been followed

Control

Intervention

Line A in Part I has been correctly identified

Control

Intervention

3(6%)

4(8%)

17 (34%)

1.000_$

0.000

2(4%)

1(2%)

3(6%)

33(66%)

1.000_$

0.000

1

3(6%)

7(14%)

3(6%)

34(68%)

1.000_$

0.000

scoring ssytem .

Only one condition per line has been identified

All the 50 post-intervention DCs in the intervention

scored more than 13 (≈60%) points with 21 DC received

a satisfactory score of ≥19 compared with no satisfac-

Control

Intervention

17(34%) 21(42%)

21(42%) 47(94%)

0.410_$

0.000

Co-morbid conditions has been identified in Part II of DC

Control 4(8%) 6 (12%)

0.741

1

35

$

0.000

Intervention

5(10%)

33(66%)

tion as it was our belief that they could not be consid-

ered as independent. For example success in the clinical

auditing feedback is dependent upon the success of the

case stimulation and the presentation.

Although further testing in other setting using large sam-

ple is needed, our intervention is generic, it is hoped that

this may serve as a model for improving the training

value of the DC completion and other residency teach-

ing sessions.

Did not utilize mechanistic terminal events

Control

Intervention

10(20%)

9(18%)

7(14%)

42(84%)

⁰^.⁴²⁴$

0.000

Did not utilize symptoms and signs

Control

Intervention

19(38%)

21(42%)

20(40%)

48 (96%)

0.680_$

0.000

Did not report abbreviations

Control

Intervention

45 (90%)

46 (92%)

44(88%)

50 (100%)

0.749

0.410

$

=

statistically significant, DC= Death certificate, Fisher’s Exact Test

(2-sided) was used in case with the count is less than 5 per cell,

Study limitations

Our study has several important limitations; firstly, in-

terns rotate between clinical departments, it was not

possible to ensure that the interns who were trained ini-

tially also participated in the post-intervention phase of

the study as some of them could have rotated out of the

department. Secondly, this was a small sampled study

limited to a unit in a tertiary-based centre, increasing the

size and testing this intervention in other settings will

increase the precision of our results, although this might

also introduce variability in the nature of the delivered

intervention. Thirdly, the study design limits our capa-

bility of testing individual participant’s performance and

the factor(s) that influence performance. Future evalua-

tion of this intervention could examine which factor (s)

influence positive change. This in turn may help tailor

future educational interventions. Lastly, we do not know

how many of the physicians that participated in part or

whole sessions of the intervention and physician prior

training on competing DC before the intervention. We

do not think that these had a major impact on the results,

as the pre-intervention and control rates were similar

and DC were randomly selected, thus limiting contami-

nants/confounders.

Discussions

Our study suggests that most completed DCs were not

competently certified with re_,g₉ard to accuracy of infor-

1

mation (MAHI parameters) entered. We found that

educational intervention which includes distribution of

educational material, presentation by a skilled instruc-

tors, fictional clinical vignette, and clinical mortality/

auditing feedback sessions resulted in 34% increase in

this core clinical practice among junior physicians. The

usual increase in physicians’ performance from similar

educational interventions is between 12-15% . The

high performance in our study could be attributed to the

multifaceted design. This sharp improvement in the ac-

curacy rates could translate into better mortality data and

improved quality of data for epidemiologic and legal

purposes. Junior physicians are performing this practice

frequently during the course of their training period.

Therefore, this educational intervention served as a rare

opportunity to bridge the gap in physicians’ training and

practice.

4-7

Previous studies have evaluated an educational interven-

4

-9

tion to improve the quality of DC . In those studies, the

rate of improvement are either insignificant, short term

or have small effect size but their study methods was

very different (no controls) from ours and in most cases

single intervention was employed. Two studies by Lak-

kireddyet al from USA and Myers and Farquhar from

Canada attempted to improve the likelihood of achiev-

ing a significant change through the using multi-prong

approach which led to significant improvement of the

results similar to our findings.

Conclusions

This multifaceted educational intervention is an effec-

tive tool to significantly improve junior physicians’ per-

formance in completing DC. Our findings are important

given increase public health concern regarding the qual-

ity of DC which is one of the principal sources of health

information and in many countries is the most reliable

form of health data.

9

7

In our study, we used multifaceted intervention cogni-

zant of the steps involved in skill and behavioral change

and the limitation of single interventions. Our analysis

used a validated scoring system and our designed is

both vertical (pre-post assessment) and horizontal

Another source of concern is the recent stipulations from

the National University Commission, MDCN, and the

postgraduate medical colleges on the need to implement

competency-based curricula in medical education. Resi-

dency programs should consider incorporating targeted

education coupled with clinical auditing feedback

(morbidity and mortality meetings, and during rounds)

into in-service training. The MDCN should also incor-

porate completion of DC as part of the houseman-ship

evaluation form and the continuing medical education

package for physicians.

9

(

intervention verse control). It could be argued, there-

fore, that the increased accuracy rates observed were not

merely as a result of time and increase experience in

practice or were due to some unforeseen intervention.

Reasonably, the fact that the effect size between assess-

ments were extremely large further supporting the con-

tribution of the intervention since the control cohort

were not given any formal presentation or interventions

as far as DC completion is concerned. We did not set out

to compare the individual components of the interven-

1

36

Authors’ contributions

Acknowledgements

GMA and MMG conceived the study, obtained ethical

approval, co-instructs all the intervention, distributed the

learning materials and drafted the manuscript. MB,

AMA, IBA and HEA co-instruct all the intervention

including the clinical auditing feedback and participated

in the data collection and analysis. All the others read

and approved the final manuscript.

We would like to appreciate Ms Nkwabzigu Ngamarju

(Gloria. A) of the Medical information and statistics unit

of University of Maiduguri Teaching Hospital with data

collection and maintenance and Mallam Balale Garba

Nafada of the ethics committee for approval of the

study. The power point presentation of the intervention

workshop can be obtained from the corresponding au-

thors if the reader wishes to use it.

Conflict of interest: None

Funding: None

References

1

2

.

Degani AT, Patel RM, Smith BE,

Grimsley E. The effect of student

training on accuracy of completion

of death certificates. Med Educ

Online [serial online]

7. Myers KA, Farquhar DR. Improv-

14. James TM, Mintz JM, McLaughlin

K. Evaluation of a multifaceted

"Resident-as-Teacher" educational

intervention to improve morning

report. BMC Medical Education

2006, 6:20 doi:10.1186/1472-6920

-6-20

15. Steele-Johnson, D. (2000). "Goal

orientation and task demand effects

on motivation, affect, and perform-

ance". Journal of Applied Psychol-

ogy85 (5): 724–738.

ing the accuracy of death certifica-

tion. CMAJ. 1998;/158:/1317-23.

8. Pain CH, Aylin P, Taub NA, Botha

JL. Death certification: production

and evaluation of a training video.

Med Educ. 1996;/30:/434-9.

9. Lakkireddy DR, Basarakodu KR,

Vacek JL, et al. Improving death

certificate completion: a trial of

two training interventions. J Gen

Intern Med. 2007;/22:/544-8.

10. Barber JB. Improving the accuracy

of death certificates. J Natl Med

Assoc. 1992;84(12):1007–8.

11. Prochaska JO, DiClemente CC:

Stages and processes of selfchange

of smoking: toward an integrative

model of change. J Consult Clin

Psychol 1983, 51:390-395.

12. Bero L, Grilli R, Grimshaw J,

Harvey E, Oxman A, Thomson M.

Closing the gap between research

and practice: an overview of sys-

tematic reviews of interventions to

promote implementation of re-

search findings. BMJ.

2

009;14:17.doi:10.3885/

meo.2009.Res00315

Rasmussen SA, Wong LY, Yang

Q, May KM, Friedman JM. Popu-

lation-based analyses of mortality

in trisomy 13 and trisomy 18.

Pediatrics. 2003;111:777–84. doi:

0

.1542/peds.111.4.777.

doi:10.1037/0021-9010.85.5.724.

PMID 11055145.

3

4

.

Arias E, Smith BL. Deaths: pre-

liminary data for 2001. Natl Vital

Stat Rep. 2003;51:1–44.

Maudsley G, William EM. Death

certification by house officers and

general practitioners-practice and

performance. J Public Health

Med. 1993;15:192–201.

Villar J, Pérez-Méndez L. Evaluat-

ing an educational intervention to

improve the accuracy of death

certification among trainees from

various specialties. BMC Health

Serv Res. 2007; 7: 183.

16. The ICDF-10 Classification of

Diseases and related problems (2

nd

Edition). Geneva, World Health

Organization, 2004.

17. Cohen J: Statistical power analysis

for the behavioural sciences. 2nd

edition. New York: Academic

Press; 1988.

5

.

Doi:10.1186/1472-6963-7-183

Weeramanthri T, Beresford W,

Sathianathan V. An evaluation of

an educational intervention to

improve death certification prac-

tice. Aust Clin Rev. 1993;/13:/185-

1998;317:465–8.

6

.

13. Smith W. Evidence for the effec-

tiveness of techniques to change

physician behavior. Chest.

2000;118:8S–17S.

9

.